[1] "year" "Area" "Seas"
[4] "Bio_smry" "SpawnBio" "Recruit_0"
[7] "retainB_1" "retainN_1" "retainB_2"
[10] "retainN_2" "retainB_3" "retainN_3"
[13] "retainB_4" "retainN_4" "retainB_5"
[16] "retainN_5" "deadB_1" "deadN_1"
[19] "deadB_2" "deadN_2" "deadB_3"
[22] "deadN_3" "deadB_4" "deadN_4"
[25] "deadB_5" "deadN_5" "F_1"
[28] "F_2" "F_3" "F_4"
[31] "F_5" "SPRratio" "rec_dev"
[34] "raw_rec_dev" "model_run" "iteration"
[37] "scenario" "end_year" "years_until_terminal"
Comparing Future Red Tide Scenarios
For this test run, I took the SEDAR Red Grouper stock assessment and used SSMSE to introduce future red tide events in the OM and EM. There were 8 total scenarios:
- No red tide X no red tide: the base stock assessment with red tide events in 2005 and 2014.
- No red tide X 2 red tide: red tide in 2018 and 2021 in the EM.
- 2 red tide X no red tide: red tide in 2018 and 2021 in the OM, but not in the EM.
- 2 red tide X 2 red tide: red tide in 2018 and 2021 in the OM and EM, EM estimated.
- 2 red tide X 2 red tide fixed: red tide in 2018 and 2021 in the OM and EM, EM fixed.
- No red tide X all years red tide: no red tide in OM, all years red tide in EM.
- 2 red tide X all years red tide: red tide in 2018 and 2021 in OM, all years red tide in EM.
- red tide every 3 years X all years red tide: red tide every 3 years in OM starting in 2018, all years red tide in EM.
Import the results and summary files from the cloud.
Raw Data
Reviewing time series plots
List of the things we can plot with the ts_plot_variable function:
Here are the time series plots:
Figure 1. The fishing mortality of fleet 5 over time. This line plot demonstrates the frequency and magnitude of red tide events in the EM (black) and OM (orange) for each scenario. All 10 iterations of the OM and EM are plotted simultaneously so peaks are representative of the iteration with the highest value and not the overall trend. The OMs have fixed magnitudes and the EMs estimate a F_5 that varies in magnitude but has a median close to the fixed value in the OM.
Table 1. A summary of the Operating Model fleet 5 fishing mortality. These statistics only include values where F_5 > 0.
| Scenario | Mean | Standard Dev | Median |
|---|---|---|---|
| rt_2_x_no_rt | 0.1 | 0 | 0.1 |
| rt_2_x_rt_2 | 0.1 | 0 | 0.1 |
| rt_2_x_rt_2_fixed | 0.1 | 0 | 0.1 |
| rt_2_x_all_yrs | 0.1 | 0 | 0.1 |
| rep_3_x_all_yrs | 0.1 | 0 | 0.1 |
Table 2. A summary of the Estimated Model fleet 5 fishing mortality. These statistics only include values where F_5 > 0.
| Scenario | Mean | Standard Dev | Median |
|---|---|---|---|
| no_rt_x_rt_2 | 0.001 | 0.006 | 0.000 |
| rt_2_x_rt_2 | 0.091 | 0.017 | 0.095 |
| rt_2_x_rt_2_fixed | 0.100 | 0.000 | 0.100 |
| no_rt_x_all_yrs | 0.001 | 0.003 | 0.000 |
| rt_2_x_all_yrs | 0.007 | 0.021 | 0.000 |
| rep_3_x_all_yrs | 0.030 | 0.035 | 0.015 |
Figure 2. The biomass of fish killed by red tide over time. This line plot demonstrates the biomass removed by red tide. While fishing mortality is fixed in the OM, the biomass removed can vary depending on the available biomass.
Figure 3. The spawning stock biomass (SSB) over time. These plots best demonstrate the effects of the red tide mortality on the spawning availability, with the increased frequency and magnitude of red tides resulting in lower SSB.
Figure 4. The total retained biomass of the fishery over time. This is the sum of all columns starting with “RetainB” so it does not include discards or red tide mortality. This plot demonstrates how impactful high mortality events can reduce catch.
Reviewing derived quantities
Additional time series plots but derived quantities. Below is a list of all the derived quantity variables:
[1] "Value.SSB" "Value.Recr" "Value.SPRratio"
[4] "Value.F" "Value.Bratio" "Value.ForeCatch"
[7] "Value.OFLCatch" "Value.ForeCatchret" "Value.lnSPB"
[10] "year" "model_run" "iteration"
[13] "scenario" "end_year" "years_until_terminal"
Figure 5. Recruitment over time by scenario. Recruitment is determined by the Beverton-Holt equation using steepness, R0, and SigmaR. Steepness is fixed at 0.99. R0 and SigmaR were estimated in SEDAR 61 and SEDAR 88.
Figure 6 (same as figure 3). The spawning stock biomass (SSB) over time. These plots best demonstrate the effects of the red tide mortality on the spawning availability, with the increased frequency and magnitude of red tides resulting in lower SSB.
Figure 7. The Spawning Potential Ratio (SPR) over time by scenario. This is the spawning output with fishing:the spawning output without fishing so it is a higher value when the reproductive potential is high. SPR tends to spike when there is a red tide event.
Figure 8. The BRatio over time by scenario. The BRatio is the current SSB over the unfished SSB so the value indicates the status of the SSB relative to a unfished scenario. This can be usefull if there is a reference point for BRatio like 0.2 as a cut-off for overfished. Higher frequencies and magnitudes of red tide cause a BRatio less than 0.2 more often.
Figure 9. The total fishing mortality over time by scenario. The default scenario indicates that fishing over time has low variability. When Red tide events are introduced, the variability increases drastically as F includes red tide mortality. Since red tide is a by-catch fleet, it is included in this value.
Ratio Time Series
Fishing mortalities
My new method for plotting EM:OM involves creating separate OM and EM data frames, joining them by year, scenario, and iteration, then dividing the EM value/ OM value for every variable. That way when plotted it is truly the EM for each model run, year and iteration divided by the OM from the same year and iteration but one model run. If the om and em value are zero I changed the ratio to 1 because that means that the EM = OM.
Figure 10. Each fleet’s fishing mortality ratio (EM:OM) over time by scenario. The line indicates the median and the ribbon is the 25-75% quartiles. F_5 is consistently underestimated, it is more underestimated in future years where there is less data.
F_5
A zoomed in look at just F_5 from the previous plots.
Figure 11. Red tide mortality ratio (EM:OM) over time by scenario. The line indicates the median and the ribbon is the 25-75% quartiles. Red tide mortality is consistently underestimated, it is more underestimated in future years where there is less data.
F
All the F’s added together from the previous plots.
Figure 11B. Overall mortality ratio (EM:OM) over time by scenario. The line indicates the median and the ribbon is the 25-75% quartiles. Red tide mortality is consistently underestimated, it is more underestimated in future years where there is less data.
Version #2 using the F.Value from dqs
Biomass
Figure 12. Biomass ratio (EM:OM) over time by scenario. The line indicates the median and the ribbon is the 25-75% quartiles. Red tide mortality is consistently underestimated, it is more underestimated in future years where there is less data.
Recruitment
Figure 13. Recruitment ratio (EM:OM) over time by scenario. The line indicates the median and the ribbon is the 25-75% quartiles. Recruitment is not over or underestimated, but there are a few outliers where the EM overestimated by 10x.
SSB
Figure 14. SSB ratio (EM:OM) over time by scenario. The line indicates the median and the ribbon is the 25-75% quartiles. SSB is not over or underestimated, but it is more likely to be overestimated, the later years are overestimated. SSB may spike in response to red tide events based on the red_tide_regular_5_mortality_5 scenario.
Terminal Year
Plotting the same ratios as above but by the “time from the terminal year” instead of year. These plots use mean instead of median because there were too many “years until terminal” with no red tide which skewed to 1 and resulted in straight lines. F_5 in all of these plots was a straight line when median was used.
Fishing Mortality
Figure 15. Each fleet’s fishing mortality ratio (EM:OM) over the years from terminal year by scenario. The line indicates the mean and the ribbon is the 25-75% quartiles. F_5 is consistently underestimated, there is higher variation where there is less data.
F_5
A zoomed in look at just F_5 from the previous plots.
Figure 16. Red tide mortality ratio (EM:OM) over the years from terminal year by scenario. The line indicates the mean and the ribbon is the 25-75% quartiles. F_5 is consistently underestimated, there is higher variation where there is less data.
Biomass
Figure 17. Biomass ratio (EM:OM) over the years from terminal year by scenario. The line indicates the mean and the ribbon is the 25-75% quartiles. Red tide mortality is consistently underestimated, it is more underestimated in years where there is less data.
Recruitment
Figure 18. Recruitment ratio (EM:OM) over time by scenario. The line indicates the mean and the ribbon is the 25-75% quartiles. Recruitment is overestimated in the terminal year of the model run.
SSB
Figure 19. SSB ratio (EM:OM) over the years from the terminal year by scenario. The line indicates the mean and the ribbon is the 25-75% quartiles. SSB is underestimated when red tide is introduced, with higher variability in the terminal year of the default model.
Management
Term Plots
Figure 20. Term plots are meant to be a “short term” (2000-2030) and “long term” (1989-2067) look at a few key parameters from the time series dataset. These plots demonstrate that red tide events correspond with SPR Ratio spikes, and can slowly decrease retained biomass or spawning biomass in the short term. In the long term, there are no trends.
Management Term Plots
Figure 21. Management Term plots are meant to be a “short term” (2000-2027) and “long term” (1989-2067) look at a few key parameters from the time series dataset. These plots demonstrate that red tide events correspond with SPR Ratio spikes, and can slowly decrease retained biomass or spawning biomass in the short term. In the long term, there are no trends. I tried to add median trend lines for the OM and EM but they aren’t very clear because of how the lines are colored.
Relative Error Plots
Inspired by Wetzel and Punt et al. 2011
I am going to attempt to make their Relative Error Plots.
RE = (E - T) / T
If the RE is NA (T = 0), the RE = 0. This introduces issues in the _all year scenarios where many of the T values are zero and the E > 0.
Warning: Removed 3300 rows containing non-finite outside the scale range
(`stat_boxplot()`).
Figure 21. Relative error plots for each variable of interest zoomed in.
Figure 22. Relative error plots of each variable of interest on the same patchwork.
Figure 23. Relative error plots of each variable of interest with fixed x-limits between -2 and 5.
Red Tide Magnitudes and Frequencies
The sums and counts of each red tide treatment to see which are comparable.
| scenario | n_years | F_5_sum | catch_sum |
|---|---|---|---|
| rt_2_x_no_rt | 2 | 0.000000 | 0.00 |
| rt_2_x_rt_2 | 2 | 18.776224 | 236704.35 |
| rt_2_x_rt_2_fixed | 2 | 18.999459 | 241566.99 |
| rt_2_x_all_yrs | 2 | 16.959422 | 219438.45 |
| rep_3_x_all_yrs | 10 | 48.993794 | 659319.08 |
| no_rt | NA | 0.000000 | 0.00 |
| no_rt_x_rt_2 | NA | 1.101803 | 15755.75 |
| no_rt_x_all_yrs | NA | 2.999020 | 47008.46 |
Table 3. The magnitudes and frequencies of each scenario with the summarized catch to demonstrate scale differences.
MAPE
Mean Average Percentage Error
Notes: When there is no red tide in the OM, a matching EM results in less error in 2018 and a higher error in 2021. When there is a red tide in the OM, a matching EM results in less error in 2018 and more error in 2021.
Misspecifying the model does not increase the error, instead there is more error in 2021 and less error in 2018.
Notes: When there is no red tide in the OM, a matching EM results in less error in 2018 and a higher error in 2021. When there is a red tide in the OM, a matching EM results in less error in 2018 and more error in 2021.
Misspecifying the model does not increase the error, instead there is more error in 2021 and less error in 2018.
Relative Error
Warning: Removed 1600 rows containing non-finite outside the scale range
(`stat_summary()`).
Removed 1600 rows containing non-finite outside the scale range
(`stat_summary()`).
Warning: Removed 4080 rows containing non-finite outside the scale range
(`stat_summary()`).
Removed 4080 rows containing non-finite outside the scale range
(`stat_summary()`).
Warning: Removed 3400 rows containing non-finite outside the scale range
(`stat_summary()`).
Removed 3400 rows containing non-finite outside the scale range
(`stat_summary()`).
Warning: Removed 8640 rows containing non-finite outside the scale range
(`stat_summary()`).
Removed 8640 rows containing non-finite outside the scale range
(`stat_summary()`).
Warning: Removed 6180 rows containing non-finite outside the scale range
(`stat_summary()`).
Removed 6180 rows containing non-finite outside the scale range
(`stat_summary()`).
Other Explorations
Exploring Recruitment
“rec_dev” “raw_rec_dev” “Recruit_0”
Ro Exploration
Extracted the R0 from SR_LN_R0 from the summary$scalar.
Testing new plots
MAPE
Relative Error
Ratio EM:OM
# A tibble: 16 × 4
scenario year_type ee_F ee_removals
<fct> <chr> <dbl> <dbl>
1 no_rt No Red Tide 0.295 0.686
2 no_rt Red Tide 0.959 3.72
3 no_rt_x_rt_2 No Red Tide 0.320 0.772
4 no_rt_x_rt_2 Red Tide 1.26 Inf
5 rt_2_x_no_rt No Red Tide 0.938 1.48
6 rt_2_x_no_rt Red Tide 12.5 58.1
7 rt_2_x_rt_2 No Red Tide 0.344 0.569
8 rt_2_x_rt_2 Red Tide 2.15 9.76
9 rt_2_x_rt_2_fixed No Red Tide 0.319 0.654
10 rt_2_x_rt_2_fixed Red Tide 0.993 4.14
11 no_rt_x_all_yrs No Red Tide 0.415 Inf
12 no_rt_x_all_yrs Red Tide 1.02 Inf
13 rt_2_x_all_yrs No Red Tide 0.587 Inf
14 rt_2_x_all_yrs Red Tide 3.64 15.7
15 rep_3_x_all_yrs No Red Tide 3.61 Inf
16 rep_3_x_all_yrs Red Tide 4.23 17.8
Warning: Removed 800 rows containing non-finite outside the scale range
(`stat_boxplot()`).
Warning: Removed 40 rows containing non-finite outside the scale range
(`stat_boxplot()`).
Testing more plots
Warning: Removed 1600 rows containing non-finite outside the scale range
(`stat_summary()`).
Removed 1600 rows containing non-finite outside the scale range
(`stat_summary()`).
Coordinate system already present. Adding new coordinate system, which will
replace the existing one.
Coordinate system already present. Adding new coordinate system, which will
replace the existing one.
Testing ratios
I think something is off about the EM:OM ratio in the previous plots so I’m going to try using the ratio_df from earlier.
Warning: Removed 61 rows containing non-finite outside the scale range
(`stat_summary()`).
Warning: Removed 800 rows containing non-finite outside the scale range
(`stat_summary()`).
New content
stat of variables over time plots
dq of variables over time plots
Threshold Table
My goal with these tables is to test setting a “threshold” of accepted deviation from the mean. In this case, I chose 10%, so a model_run or scenario is flagged when the EM is within 10% of the OM value. This could help when arguing for a “best” model and the threshold and variable could be changed depending on our goals.
`summarise()` has grouped output by 'scenario'. You can override using the
`.groups` argument.
| scenario | is_within_10_percent | scenario_sum |
|---|---|---|
| no_rt | FALSE | 4 |
| no_rt | TRUE | 616 |
| no_rt_x_rt_2 | FALSE | 24 |
| no_rt_x_rt_2 | TRUE | 596 |
| rt_2_x_no_rt | FALSE | 28 |
| rt_2_x_no_rt | TRUE | 592 |
| rt_2_x_rt_2 | FALSE | 9 |
| rt_2_x_rt_2 | TRUE | 611 |
| rt_2_x_rt_2_fixed | FALSE | 3 |
| rt_2_x_rt_2_fixed | TRUE | 617 |
| no_rt_x_all_yrs | FALSE | 303 |
| no_rt_x_all_yrs | TRUE | 317 |
| rt_2_x_all_yrs | FALSE | 293 |
| rt_2_x_all_yrs | TRUE | 327 |
| rep_3_x_all_yrs | FALSE | 288 |
| rep_3_x_all_yrs | TRUE | 332 |
`summarise()` has grouped output by 'year', 'scenario'. You can override using
the `.groups` argument.
| year | scenario | Is within 10% | Is outside 10% |
|---|---|---|---|
| 2018 | no_rt | 10 | NA |
| 2018 | no_rt_x_rt_2 | NA | 10 |
| 2018 | rt_2_x_no_rt | NA | 10 |
| 2018 | rt_2_x_rt_2 | 8 | 2 |
| 2018 | rt_2_x_rt_2_fixed | 10 | NA |
| 2018 | no_rt_x_all_yrs | NA | 10 |
| 2018 | rt_2_x_all_yrs | 6 | 4 |
| 2018 | rep_3_x_all_yrs | 2 | 8 |
| 2021 | no_rt | 10 | NA |
| 2021 | no_rt_x_rt_2 | NA | 10 |
| 2021 | rt_2_x_no_rt | NA | 10 |
| 2021 | rt_2_x_rt_2 | 3 | 7 |
| 2021 | rt_2_x_rt_2_fixed | 10 | NA |
| 2021 | no_rt_x_all_yrs | NA | 10 |
| 2021 | rt_2_x_all_yrs | 1 | 9 |
| 2021 | rep_3_x_all_yrs | 2 | 8 |
Boxplot Tests
| year | scenario | EM_2047 | OM |
|---|---|---|---|
| 2005 | no_rt | 0.215 | 0.224 |
| 2005 | no_rt_x_rt_2 | 0.214 | 0.224 |
| 2005 | rt_2_x_no_rt | 0.223 | 0.224 |
| 2005 | rt_2_x_rt_2 | 0.217 | 0.224 |
| 2005 | rt_2_x_rt_2_fixed | 0.214 | 0.224 |
| 2005 | no_rt_x_all_yrs | 0.215 | 0.224 |
| 2005 | rt_2_x_all_yrs | 0.216 | 0.224 |
| 2005 | rep_3_x_all_yrs | 0.219 | 0.224 |
| 2014 | no_rt | 0.166 | 0.181 |
| 2014 | no_rt_x_rt_2 | 0.165 | 0.181 |
| 2014 | rt_2_x_no_rt | 0.207 | 0.181 |
| 2014 | rt_2_x_rt_2 | 0.171 | 0.181 |
| 2014 | rt_2_x_rt_2_fixed | 0.166 | 0.181 |
| 2014 | no_rt_x_all_yrs | 0.167 | 0.181 |
| 2014 | rt_2_x_all_yrs | 0.171 | 0.181 |
| 2014 | rep_3_x_all_yrs | 0.172 | 0.181 |
| 2018 | no_rt | 0.000 | 0.000 |
| 2018 | no_rt_x_rt_2 | 0.000 | 0.000 |
| 2018 | rt_2_x_no_rt | 0.000 | 0.100 |
| 2018 | rt_2_x_rt_2 | 0.095 | 0.100 |
| 2018 | rt_2_x_rt_2_fixed | 0.100 | 0.100 |
| 2018 | no_rt_x_all_yrs | 0.000 | 0.000 |
| 2018 | rt_2_x_all_yrs | 0.085 | 0.100 |
| 2018 | rep_3_x_all_yrs | 0.080 | 0.100 |
| 2021 | no_rt | 0.000 | 0.000 |
| 2021 | no_rt_x_rt_2 | 0.003 | 0.000 |
| 2021 | rt_2_x_no_rt | 0.000 | 0.100 |
| 2021 | rt_2_x_rt_2 | 0.087 | 0.100 |
| 2021 | rt_2_x_rt_2_fixed | 0.100 | 0.100 |
| 2021 | no_rt_x_all_yrs | 0.000 | 0.000 |
| 2021 | rt_2_x_all_yrs | 0.074 | 0.100 |
| 2021 | rep_3_x_all_yrs | 0.065 | 0.100 |
Histogram Tests
The OM mean is the solid line, the EM mean is the dashed line.
Absolute Error Test
Mean Absolute Error (MAE)
2047
Symmetric Mean Absolute Percentage Error (SMAPE)
2047
More Variables Absolute Error
Residual Plots
Long Term Residual Sums (1980-2050)
| Scenario | Commercial Catch Residual Sum | Recreational Catch Residual Sum | Red Tide Discards Residual Sum |
|---|---|---|---|
| no_rt | 29 | -880 | -5073 |
| no_rt_x_rt_2 | -32 | -1220 | -5107 |
| rt_2_x_no_rt | -385 | -1900 | -21309 |
| rt_2_x_rt_2 | 246 | -665 | -6094 |
Short Term Residual Sums (2017-2027)
| Scenario | Commercial Catch Residual Sum | Recreational Catch Residual Sum | Red Tide Discards Residual Sum |
|---|---|---|---|
| no_rt | -230 | -619 | 0 |
| no_rt_x_rt_2 | -360 | -869 | 371 |
| rt_2_x_no_rt | 1828 | 3406 | -25170 |
| rt_2_x_rt_2 | -6 | -152 | -2430 |
Long Term Residual Sums (1980-2026)
| Scenario | Commercial Catch Residual Sum | Recreational Catch Residual Sum | Red Tide Discards Residual Sum |
|---|---|---|---|
| no_rt | 189 | 23 | -8648 |
| no_rt_x_rt_2 | 174 | -12 | -4898 |
| rt_2_x_no_rt | -278 | -237 | -14151 |
| rt_2_x_rt_2 | 56 | 0 | 1135 |
Short Term Residual Sums (2017-2027)
| Scenario | Commercial Catch Residual Sum | Recreational Catch Residual Sum | Red Tide Discards Residual Sum |
|---|---|---|---|
| no_rt | -225 | -471 | 0 |
| no_rt_x_rt_2 | -443 | -876 | 2824 |
| rt_2_x_no_rt | 1916 | 3497 | -25170 |
| rt_2_x_rt_2 | 19 | 35 | 115 |